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WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

• Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 
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DETAILED ACTION 

Priority 

1. Receipt is acknowledged of papers submitted under 35 U.S.C. 1 19(a)-(d), which papers 
have been placed of record in the file. 

Information Disclosure Statement 

2. The references listed in the Information Disclosure Statement submitted on 03/18/2004 
have been considered by the examiner (see attached PTO-1449). 

Specification and Drawing 

3. The disclosure is objected to because of the following: 

a. on page 3, lines 2, 15, 18 the term "digital sung signal" is unclear or confused. Is 
that means "digital sound signal", or something else? (similar term is also found on page 4, lines 
4 andl4; page 7, lines 10-11; page 1, line 1 1). Appropriate correction/clarification is required. 

b. on page 5, lines 28-32, the stated feature "The mixer means 4 therefore replace 
the fundamental frequency and the harmonics of the voice signal by the fundamental frequency 
and the harmonics of each of the notes of the music signal during the note" lacks reasonably 
clear/specific description for implementing it, because the disclosure of two proportions of 
20ms-frame speech and music signals in the last step/mechanism (see lines 23-27) is only one 
time domain characteristic, and the range of fundamental frequency is much narrower that that of 
music. Therefore, the disclosure is inefficient to enable one of ordinary skill in the art to replace 
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fundamental frequency and the harmonics from that of speech to that music. Appropriate 
correction/clarification is required. 

c. between page 5, lines 28-32 and page 6, lines 1-2, the statement "replace the 
fundamental frequency and the harmonics of the voice signal by the fundamental frequency and 
the harmonics of each of the notes of the music signal during the note" directly conflicts with the 
statement "a proportion Y% of a musical sinusoidal signal deduced from the signal S2 is 
substituted for a proportion X% of the speech sinusoidal signal." Appropriate correction is 
required. 

d. on page 6, lines 29-34, the statement "said (a) power peak corresponding to the 
fundamental frequency of a vowel" is not true, since a power peak only indicates one of 
harmonics and normally not the fundamental frequency. In order to find a fundamental 
frequency, more process(es) need to be done, but the applicant did not disclose them. 
Appropriate correction/clarification is required. 

e. on page 6, line 35 to page 7, lines 5, it says that "the vocoder 6 of the terminal 
includes a voice activity detector (VAD) ..." and "if the amplitude of the voice signal tends 
towards zero, the VAD may force the mixer means to move on to the next note of the score"; 
meanwhile, Fig. 1 shows that the mixer 4 is processed the signals before the vocoder 6. 
Therefore, the VAD (in the vocoder) forcing the mixer is logically/structurally incorrect, or not 
enable. Appropriate correction is required. 

f The drawings (Figs 1-5) are objected to because all the text labels (legends) 
regarding the function blocks in Figs 1-5 are missing. In this case, the applicant is required to 
provide text label(s) for each functional block in the drawings, based on the disclosure in the 
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specification and associated with the reference number, for the propose of better understanding 
the claimed invention and comparing the different structures between the disclosed drawings 
and/or prior art drawings. (Also see 37 CFR 1 .84 (o)). 

Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to 
the Office action to avoid abandonment of the application. Any amended replacement drawing 
sheet should include all of the figures appearing on the immediate prior version of the sheet, 
even if only one figure is being amended. The figure or figure number of an amended drawing 
should not be labeled as "amended." If a drawing figure is to be canceled, the appropriate figure 
must be removed from the replacement sheet, and where necessary, the remaining figures must 
be renumbered and appropriate changes made to the brief description of the several views of the 
drawings for consistency. Additional replacement sheets may be necessary to show the 
renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an 
application must be labeled in the top margin as either "Replacement Sheet" or "New Sheet" 
pursuant to 37 CFR 1 .121(d). If the changes are not accepted by the examiner, the applicant will 
be notified and informed of any required corrective action in the next Office action. The 
objection to the drawings will not be held in abeyance. 

It is also reminded that there is no new subject matter allowed in any amended 
specification and drawing. 



Claim Rejections - 35 USC§112 
The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 
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The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject 
matter which the applicant regards as his invention. 

4. Claims 1-21 are rejected under 35 U.S.C. 1 12, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. 

Regarding claims 1 and 1 1, the claimed term "digital sung signal" (also included in 
claims 5-6, 10, 12, 16-17, 21) is indefinite, because it is unclear what the term really is. It is 
noted that "digital sung signal" is not commonly accepted/used term in the art and lacks clear 
definition/description in the specification. 

Regarding claims 2-10 and 12-21, the rejection is based on the same reason as described 
for claims 1 and 11, because the dependent claims include or inherit the same or similar 
problematic limitation as their parent claim(s). 

5. The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of making 
and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it 
pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode 
contemplated by the inventor of carrying out his invention. 

6. Claims 3-4 and 14-15 are rejected under 35 U.S.C. 112, first paragraph, as failing to 
comply with the enablement requirement. The claim(s) contains subject matter which was not 
described in the specification in such a way as to enable one skilled in the art to which it pertains, 
or with which it is most nearly connected, to make and/or use the invention. 

Regarding claims 3 and 14, the claimed limitation "means for replacing the fundamental 
frequency of said speech signal by the fundamental frequency associated with a note of said 
music signal" lacks enablement to one of ordinary skill in the art based on the disclosure of the 
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specification (see the closet disclosure of the specification: page 5, line 28 to page 6, line 34). It 
is noted that no where in the specification specifically describes how to obtain/extract a 
fundamental frequency of the speech signal, and the disclosure "said power peak corresponding 
to the fundamental frequency of a vowel" (page 6, line 3 1-32) is incorrect and inefficient to 
obtain a fundamental frequency of the signal. Further, it is well known in the art that the range 
of fundamental frequencies of human speech is much narrower than that of musical instruments, 
so that the replacement cannot be easily implemented by one of ordinary skill in the art without 
solving this problem; otherwise, the replacement cannot be enabled or lacks meaningful 
operation. There is no evidence in the specification to solve this problem to one of ordinary skill 
in the art. Furthermore, it is noted that the claim limitation conflicts with the specification 
disclosure that states "a proportion Y% of a musical sinusoidal signal deduced from the signal 
S2 is substituted for a proportion X% of the speech sinusoidal signal" (page 5, line 35 to page 6, 
line 2) (meaning musical signal is placed by speech). Therefore, the claim contains subject 
matter which was not described in the specification in such a way as to enable one skilled in the 
art to make and/or use the invention, without undue experimentation. 

Regarding claims 3 and 15, the rejection is based on the same reason as described for 
claims 1 and 14, because the dependent claims include or inherit the same or similar problematic 
limitation as their parent claim(s). 



Claim Rejections - 35 USC §103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such 
that the subject matter as a whole would have been obvious at the time the invention was made to a person having . 
ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in 
which the invention was made. 

7. Claims 1-7 and 10 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
PAW ATE et al. (US 5,641,927) hereinafter referenced as PAW ATE in view of BOSS et al. (US 
5,915,237) hereinafter referenced as BOSS. 

As per claim 1, as best understood in view of the rejection under 35 USC 1 12 2 nd (see 
above), PAW ATE discloses 'autokeying for musical accompaniment playing apparatus (audio 
device) 5 (title), comprising: 

"input by the user of said audio device of an analog speech signal" (Fig.2, 'user's vocal'), 

"converting said analog [speech] signal into a digital [speech] signal comprising at least 
one fundamental frequency" (Fig. 2 shows a microphone and 'pitch (corresponding to 
fundamental frequency) estimator 23'), 

"storing a set of coded data [representing a musical score comprising a set of notes, each 
note being defined by a fundamental frequency, a duration, and an instrument that plays said 
note]" (col. 2, lines 54-67, 'the key (corresponding to pitch) of the music may also be stored in 
the CD data (set of coded data) field so not have to be computed'), 

"extracting a digital music signal from said set of coded data" (col. 2, lines 54-67, 'the 
pitch estimated and averaged from the original artist's voice (musical signal), or key 
(corresponding to pitch) from the background music or that from the CD data field is compared 
(necessarily extracting music from the related data)'), and 

"mixing a first portion of said digital [speech] signal and a first portion of said digital 
music signal to produce a digital [sung] signal" (col. 3, lines 1-40, 'change the key (portion of 
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music) of background music' and 'output (produce) to the mixer 13a to add the user's vocal 
(portion of the input digital signal; also see Figs, 2 and 2a). 

It is noted that PAW ATE does not expressly disclose the input digital signal being 
"speech signal" and the coded data "representing a musical score comprising a set of notes, each 
note being defined by a fundamental frequency, a duration, and an instrument that plays said 
note". However, this feature is well known in the art as evidenced by BOSS who discloses 
'representing speech using MIDI (musical instrument digital interface)' (title), comprising well 
known feature of MIDI data for generating music including 'identifying a musical instrument 
(i.e. piano, clarinet) for music generation, turning on a note (reflecting a musical score) or 
altering a parameter in order to generating or control sound' (col. 2, lines 7-30), which 
necessarily/inherently includes parameter data of pitch (corresponding to fundamental 
frequency) and the related time stamps (corresponding to duration) for the music note; 'encoding 
a digitized speech into a standard digital format, such as MIDI' (col. 2, lines 67 to col. 3, line 5); 
and using 'a MIDI compatible signal' for processing and storing 'speech segments' in 'the 
phoneme dictionary' including parameters of 'pitch' and 'duration' (col. 5, line 45 to col. 6, line 
28). BOSS also discloses 'mixer 204, ...receives a digitized speech signal... and a digitized 
music signal. . . and mixes the two signals together to form a single audio output' (col. 13, lines 
30-45), which further supports the claim rejection. Therefore, it would have been obvious to one 
of ordinary skill in the art at the time the invention was made to modify PAW ATE by providing 
a compatible standard digital format, such as MIDI, for representing a speech and/or music 
signals, as taught by BOSS, for the purpose (motivation) of efficiently representing, storing and 
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transmitting sound/audio signal(s) including music and/or speech signals (BOSS: col. 2, lines 27- 
29 and 56-61; col. 13, lines 58-61). 

As per claim 2 (depending on claim 1), PAW ATE in view of BOSS further discloses "a 
digital signal processor comprising said means for mixing said first portions of said digital 
speech signal and said digital music signal" (PAW ATE: col. 2, line 45 and Fig. 1, block 13). 

As per claim 3 (depending on claim 1), as best understood in view of the rejection under 
35 USC 1 12 2 nd and 1 st (see above), it is noted that the combined references disclose using the 
mismatch between the two estimated pitches (corresponding to fundamental frequencies) of user 
and reference (background music) to change (substantially replace) the key (or pitch) of 
background music (PAW ATE : col. 3, lines 1-16 and Figs. 2-2a), which is different from 
claimed "replacing the fundamental frequency of said speech signal by the fundamental 
frequency associated with a note of said music signal." However, it would have been obvious to 
one of ordinary skill in the art to use the same mismatch to change (replace) user's pitch instead 
of reference's pitch in the same manner, so as to produce the predicable result of the user's 
speech with a characteristic of the reference's (music's) pitch. It is noted that estimating and 
comparing pitches uses the same known technique (as taught by PAW ATE), and changing 
(replacing) pitch from one to the other (i.e. from reference's pitch to user's pitch, or from user's 
pitch to reference's pitch) uses in the same known method, so that, one of ordinary skill in the art 
would have recognized that solving the difference based on the teachings of PAW ATE in view 
of BOSS would have been obvious and the result would have been predicable as stated above. 

As per claim 4 (depending on claim 3), as best understood in view of the rejection under 
35 USC 1 12 2 nd and 1 st (see above), PAW ATE in view of BOSS further discloses "said 
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fundamental frequency of said speech signal is replaced by said fundamental frequency 
associated with said note of said music signal during a period substantially equal to the duration 
of said note" (BOSS: col. 5, lines 1-47, since the speech encoded into MIDI compatible signal, 
the time stamp (inherent feature reflecting duration) of a note in music could be easily used to 
associate with the related phoneme duration, as claimed). 

As per claim 5 (depending on claim 1), as best understood in view of the rejection under 
35 USC 1 12 2 nd (see above), PAW ATE in view of BOSS further discloses "adding to said digital 
[sung] signal a second portion of said digital speech signal" (PAW ATE: col. 3, line 8 'to add the 
user's vocal' reads on second portion of said digital speech signal; col. 4, lines 13-56, 'envelop', 
'residual' and 'lpc' can also be broadly interpreted as second portion of said digital speech 
signal). 

As per claim 6 (depending on claim 1), as best understood in view of the rejection under 
35 USC 1 12 2 nd (see above), PAW ATE in view of BOSS further discloses "adding to said digital 
[sung] signal a second portion of said digital music signal" (PAW ATE: col. 3, lines 1-16, 
wherein other music portions excluding key (or pitch) can be broadly interpreted as second 
portion of said digital music signal). 

As per claim 7 (depending on claim 1), PAW ATE in view of BOSS further discloses 
"replacing at least one harmonic frequency of said fundamental frequency of said speech signal 
with a harmonic frequency of said fundamental frequency associated with a note of said musical 
signal" (PAW ATE: col. 8, lines 1-4, 'indicate second or third harmonic'; BOSS: col. 6, lines 29- 
53, 'measure the pitch of the phoneme represented by the received phoneme pattern by. . . spectral 
compression and harmonic matching method'; col. 7, line 25 to col. 8, line26, 'MIDI standard' 
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that inherently includes parameter for timbre (corresponding to harmonic)'; one of ordinary skill 
in the art would have recognized that the result of matching harmonic(s) could be used for 
changing (or replacing) certain harmonic(s) of the user or reference (music) in the same/similar 
way as for changing/replacing pitch, so that the output audio (result) would have a sound 
characteristic of harmonic(s) of the music (achieving predicable result), and vice versa). 

As per claim 10 (depending on claim 1), as best understood in view of the rejection under 
35 USC 1 12 2 nd (see above), PAW ATE in view of BOSS further discloses "a vocoder for coding 
said [sung] signal" (PAW ATE: col. 3, line 63, 'phase vocoder'; col. 4, lines 47-61, 'residual 
resampling method' with 'LPC (vocoder feature); BOSS: 'the MIDI speech signal output.. .may 
transmitted over ...wireless communication, or telephone lines', so that one of ordinary skill in 
the art would have recognized that coding the processed signal would be the same as coding 
normal speech signal by using a vocoder). 

8. Claim 8 is rejected under 35 U.S.C. 103(a) as being unpatentable over PAW ATE in view 
BOSS applied to claim 1, and further in view of KAGEYAMA et al. (US 5,857,171) hereinafter 
referenced as KAGEYAMA. 

As per claim 8 (depending on claim 1), even though PAW ATE in view of BOSS 
discloses "mixing said first portions of said digital speech signal and said digital music signal" as 
stated above (see claim 1), PAW ATE in view of BOSS does not expressly disclose 
"discriminating a consonant from a vowel in said digital speech signal" and adapted to activate 
the mixing. However, the feature is well known in the art as evidenced by KAGEYAMA who 
discloses 'a vowel/consonant separator 40 (discriminator)' so that 'the consonant and vowel 
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components can be separated (discriminated) from each other by detecting a fundamental 
frequency' and 'the vowel synthesizer 43 generates the vowel signal at the pitch specified by the 
pitch calculator based on the phoneme data distributed by the phoneme data register 48' (Fig.2 
and col. 6, line 60 to col. 7, line 52), and teaches that 'the phoneme data track of the song data 
records only the vowel data of the original or model signer. . . ', which suggests that the system is 
adapted to activate a mechanism for mixed signal (after envelope generator 44, Fig 2) during 
detection of the vowel, as claimed. Therefore, it would have been obvious to one of ordinary 
skill in the art at the time the invention was made to modify PAW ATE in view of BOSS by 
providing a mechanism of separating vowel/consonant and activating a mixing process during 
detection of vowel, as taught by KAGEYAMA, for the purpose (motivation) of creating a 
harmony voice having a tone other than that of user (actual player, or karaoke singer) 
(KAGEYAMA: col. 1, lines 31-32). 

9. Claim 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over PAW ATE in view 
BOSS applied to claim 1, and further in view of KAGEYAMA et al. (US 5,712,437) hereinafter 
referenced as KAGEYAMA2. 

As per claim 9 (depending on claim 1), PAW ATE in view of BOSS does not expressly 
disclose "a voice activity detector controlling said means for mixing said first portions of said 
digital speech signal and said digital music signal." However, the feature is well known in the 
art as evidenced by KAGEYAMA2 who discloses 'if the detected state of the signing 
performance indicates a no voice period', some functions/structures 'are disabled' (col. 5, lines 
43-55), which suggests that system has a mechanism of detecting voice activity (so as being a 
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voice activity detector) and determining whether or not a function/component is disabled (so as 
controlling the function/component). Therefore, it would have been obvious to one of ordinary 
skill in the art at the time the invention was made to combine the mixing means disclosed by 
PAW ATE and BOSS with a mechanism of detecting voice activity for controlling certain 
function/component as taught by KAGEYAMA2, for the purpose (motivation) of generating a 
harmony audio signal containing an additional harmony part and/or determining to stop(or start) 
to harmony sound generation (KAGEYAMA2: abstract and col. 5, lines 54-55). 

10. Claims 11-18 and 21 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
PAW ATE in view BOSS applied to claim 1, and further in view of TANIGUCHI et al. (US 
5,712,437) hereinafter referenced as TANIGUCHI. 

As per claim 11, as best understood in view of the rejection under 35 USC 1 12 2 nd (see 
above), the rejection is based on the same reason described for claim 1 because the claim recites 
the same or similar limitations as claim 1, except the preamble limitation "a telecommunication 
terminal". However, the feature is well known in the art as evidenced by TANIGUCHI who 
discloses 'music player applicable to portable telephone terminal' (title), comprising 'portable 
telephone terminal (a telecommunication terminal) incorporating a music player device', 'CPU' 
and 'speech processor 5 for 'coding/decoding on speech signals' and producing 'hold sound 
mixed with received speech' (col. 3, line 30 to col. 4, line 1 1). Therefore, it would have been 
obvious to one of ordinary skill in the art at the time the invention was made to modify 
PAW ATE in view BOSS by providing a portable telephone terminal and mixing sound with 
speech, as taught by TANIGUCHI, for the purpose (motivation) of generating BGM 



Application/Control Number: Page 14 

10/802,835 

Art Unit: 2626 

(background music) mixed with received speech signals for the system (TANIGUCHI: col. 16, 
lines 34-56). 

In addition, in another view of teachings of PAW ATE and BOSS, since PAW ATE in 
view BOSS discloses using 'a computer system' implementing the MIDI encoding/decoding 
systems and including 'a modem for communicating with one or more other computers via the 
internet, telephone lines or other transmission medium 1 (BOSS: col. 11, line 58 to col. 12, line 
16), the computer system can be broadly interpreted as claimed "a telecommunication terminal". 
This means that the disclosure by PAW ATE in view BOSS can also satisfy the claim for the 
rejection, based on broadest reasonable interpretation of the claim in light of the specification. 

As per claim 12 (depending on claim 1 1), PAW ATE in view of BOSS and TANIGUCHI 
further discloses "transmitting said digital sung (sound) signal to another terminal in real time" 
(TANIGUCHI: col. 9, lines 14-30). 

Regarding claims 13-18 and 21 (depending on claim 1 1), the rejection is based on the 
same reason described for claims 2-7 and 10, because the claims recites the same or similar 
limitations as claims 2-7 and 10 respectively. 

1 1 . Claim 19 is are rejected under 35 U.S.C. 103(a) as being unpatentable over PAW ATE in 
view of BOSS and TANIGUCHI applied to claim 1 1, and further in view of KAGEYAMA. 

Regarding claim 19 (depending on claim 1 1), the rejection is based on the same reason 
described for claim 8, because the claims recites the same or similar limitations as claim 8. 
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12. Claim 20 is rejected under 35 U.S.C. 103(a) as being unpatentable over PAW ATE in 
view BOSS and TANIGUCHI applied to claim 1 1, and further in view of KAGEYAMA2. 

Regarding claim 20 (depending on claim 1 1), the rejection is based on the same reason 
described for claim 9, because the claims recites the same or similar limitations as claim 9. 



Conclusion 

13. Please address mail to be delivered by the United States Postal Service (USPS) as 
follows: 

Mail Stop 

Commissioner for Patents 

P.O. Box 1450 

Alexandria, VA 22313-1450 
or faxed to: 571-273-8300, (for formal communications intended for entry) 
Or: 571-273-8300, (for informal or draft communications, and please label 
"PROPOSED" or "DRAFT") 
If no Mail Stop is indicated below, the line beginning Mail Stop should be omitted from the 
address. 

Effective January 14, 2005, except correspondence for Maintenance Fee payments, 
Deposit Account Replenishments (see 1.25(c)(4)), and Licensing and Review (see 37 CFR 5.1(c) 
and 5.2(c)), please address correspondence to be delivered by other delivery services (Federal 
Express (Fed Ex), UPS, DHL, Laser, Action, Purolater, etc.) as follows: 

U.S. Patent and Trademark Office 

Customer Window, Mail Stop 

Randolph Building 

Alexandria , VA 22314 
Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Qi Han whose telephone numbers is (571) 272-7604. The 
examiner can normally be reached on Monday through Thursday from 9:00 a.m. to 7:30 p.m. If 
attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Richemond Dorvil, can be reached on (571) 272-7602. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Inquiries regarding the status of submissions 
relating to an application or questions on the Private PAIR system should be directed to the 
Electronic Business Center (EBC) at 866-217-9197 (toll-free) or 703-305-3028 between the 
hours of 6 a.m. and midnight Monday through Friday EST, or by e-mail at: ebc@uspto.gov. For 
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general information about the PAIR system, see http://pair-direct.uspto.gov. 
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